1. Field of the Invention
The present invention relates to the functional verification of integrated circuit hardware designs using dynamically generated pseudo-random test methodologies. In particular, the present invention provides a method and apparatus that tracks system resources in addition to the architectural state within a golden model, thus enabling instructions to be dynamically selected and generated as a function of both architectural state and system resource availability.
2. Description of the Related Art
Random test generation programs have been around since the early days of microprocessor verification. These programs automate the complex job of creating large test suites necessary for the functional verification of computer systems. From a historic perspective, functional verification has focused on two types of circuits, combinatorial and sequential. Combinatorial circuits do not contain state information, and therefore the outputs are strictly a finction of the input values. Sequential circuits contain memory elements that allow the circuit to have a current state. This makes the output values a function of both the current state and the input values. Early on, those in the industry recognized that creating purely random input values was not an adequate method of verifying sequential circuits, and a new test methodology that included the current state of the circuit was developed.
One method used to generate test patterns for processor verification that is well known in the art is to use a computer to generate a machine executable program that is assembled from instructions, control flags, and data selected from specially prepared tables. To get the expected results, the automatically-assembled test program is simulated on an existing hardware or software golden model under driver control. The test operator can change the drivers, thus changing the conditions under which the test programs are executed. This approach to testing is generally characterized as static testing, because the test patterns are assembled first, and are then executed in the simulation to get the expected results. The test pattern development process is not influenced by the intermediate machine states of the processor during the execution of the test.
While static testing is highly useful, it does have drawbacks. One obvious drawback is that tests created without knowledge of the intermediate machine states have great difficulty achieving high coverage of all the possible intermediate states. In addition, statically generated tests are restricted in that certain real-life cases can never be simulated. For example, some instructions within a statically generated instruction sequence must always be preceded by a specially-inserted data initialization instruction, which would ordinarily not appear in normal operation. Similarly, register usage can be somewhat more limited in a static simulation than it would ordinarily be in normal operation. Finally, statically generated simulations that include loops and branches must be very carefully (and therefore, artificially) constructed to avoid undesirable test results such as endless loops, branches that are never taken, and the like.
To address some of these issues, and to create a test environment that is more efficient and conducts a more robust test of very complex sequential circuits such as processors, the industry has moved toward the use of dynamically generated, biased pseudo-random test patterns. In dynamic testing, instructions are generated, all processor resources and facilities needed for executing the instruction are identified and initialized if required, the instruction is executed on a simulated processor, and the simulated processor state is updated to reflect the execution results. The process iterates, and each instruction generated at the beginning of each iteration is generated with knowledge of the processor state that resulted from the last step executed. Although there are differences in the details of how instructions are generated from test method to test method, in general, instructions are drawn from an instruction pool or tree specified by the test operator that contains the various types of instructions to be tested. In addition, many pseudo-random dynamic test generation methodologies allow the test operator to bias the instruction pool or tree, meaning that the operator can specify what percentage of the total instructions generated and simulated should be load-store instructions, what percentage should be arithmetic instructions, what percentage should be floating point instructions, and so forth. Those skilled in the art are generally familiar with the various current methods used to conduct functional verification of complex circuits using dynamically generated biased psuedo-random test patterns. Readers unfamiliar with these methods are referred to U.S. Pat. No. 5,202,889 to Aharon et al., entitled xe2x80x9cDynamic Process For The Generation Of Biased Pseudo-Random Test Patterns For The Functional Verification Of Hardware Designsxe2x80x9d and the patents and publications referenced therein, which is incorporated by reference for all purposes into this disclosure. U.S. Pat. No. 5,202,889 to Aharon is a good example of the current state of the art of pseudo-random test pattern generators and their usefulness for functional verification of complex circuits.
Dynamic testing conducted in the manner described by Aharon eliminates the requirement for the cumbersome predevelopment of data tables and code streams that is necessary to achieve a high level of confidence in static testing. Also, because instructions can be automatically and iteratively generated and then simulated using an appropriately biased instruction pool, dynamic testing can also provide greater test coverage than static testing in a shorter amount of time. Nevertheless, dynamically generated tests using psuedo-random test patterns such as that described by Aharon have drawbacks and limitations as well.
For example, generating stimuli based solely upon current machine state information may not adequately test some complex processors such as Very Long Instruction Word (xe2x80x9cVLIWxe2x80x9d) and Reduced Instruction Set Computer (xe2x80x9cRISCxe2x80x9d) processors, which may have special rules regarding usage of certain resources. Similarly, some processor architectures allow certain groups of instructions to execute at the same architectural time. Instructions that are dynamically generated and simulated in a pseudo-random test pattern must therefore adhere to the processor""s instruction grouping rules. This creates another case where the creation of input stimulus is not just a function of the current state, and therefore requires additional components in the test generator to restrict the grouping of instructions and possible operands to a given set of rules. In addition, while a dynamically generated test setup may run flawlessly on a single processor model, the same test may violate memory coherence rules when run on a multiprocessor model. Finally, the creation of highly deterministic code sequences, such as sequences that execute if then else statements and other conditional control breaks, can be problematic in dynamically generated tests. The present invention addresses the limitations of current-technology dynamic psuedo-random test pattern generators, which currently select and generate instructions based only upon current machine state. The present invention is a method and apparatus that is useful in a pseudo-random test generation setup because the present invention dynamically tracks selected system resources in a golden model, thus enabling the selection, scheduling, and simulation of instructions when generating a test to be based upon current and future system resource availability, current architectural state of the golden model, and the resource utilization rules appropriate for the architecture of the system under test. When the test generator selects an instruction for simulation on the golden model, the present invention assesses the system resources that the execution of the instruction will require as a function of time (processor cycles) and marks those resources as in use or unavailable for the appropriate amount of time. Tracking selected system resources in this manner increases the efficiency of a dynamic test generator, because the selection of instructions that require resources that are unavailable or that conflict with other pending instructions can be avoided until the resource or conflict clears. Moreover, the capability to track system resources enables the developers of dynamic pseudo-random test generators to incorporate and utilize special resource-utilization rules that some processor architectures may have, thus enabling the generation of more robust tests for these architectures. Dynamic pseudo-random test generators that include the present invention thus provide a better functional verification test of some processors, because such test generators are capable of taking into account machine state, resource availability, and instruction grouping rules in selecting and generating pseudo-random test stimuli.
The present invention is a method and apparatus that dynamically tracks the status of selected system resources of the golden model in a dynamic pseudo-random test generator that generates tests for processor systems, thus providing a more robust test because the present invention can enable the test generator""s instruction packer to dynamically select instructions for generation based upon the availability of system resources in addition to the processor state. Moreover, the present invention can generate time-based lists that show the historical usage and status of various system resources, thus allowing a comparison between the resource utilization of the golden model during the generation of a test and the resource utilization of the system under test when that test is run. The present invention includes a resource tracker that generates a hazard queue, wherein the availability of each system resource in a first set of selected system resources is identified as a function of time. The first set of selected system resources includes system resources the architectural state of the golden model and system resources that will cause a stall condition if unavailable, such as general purpose registers, special purpose registers, the load/store buffer, utilization of execution units, pending control branches, and pending exceptions. The resource tracker updates the hazard queue every processor cycle, and updates resource related data structures associated with resources in the hazard queue when the hazard queue indicates that the resource is changing state during the current processor cycle. The present invention also includes a static group of resource-related data structures that comprise information relating to the required resource utilization of each instruction supported by the processor system under test, and may also comprise resource-related rules dictated by the processor architecture. The resource tracker uses this static group of resource-related data structures to generate the hazard queue. The present invention is capable of generating two types of time-based lists that can be used to compare resource utilization in the golden model while a test was being generated to resource utilization in a system under test when the test generated is run on the system under test. The first list type indicates the status of each resource within said first set of system resources as a function of time in processor cycles, and the second list type indicates the status of other system resources that are tracked but not typically updated in resource-related data structures, such the physical memory system, virtual memory system, cache, system busses, and interrupts generated.